In this paper we consider the problem of how to measure the strength of statistical evidence from the perspective of evidence amalgamation operations. We begin with a fundamental measurement amalgamation principle : for any measurement, the inputs and outputs of an amalgamation procedure must be on the same scale, and this scale must have a meaningful interpretation vis a vis the object of measurement. Using the p value as a candidate evidence measure, we examine various commonly used approaches to amalgamation (...) of evidence across similar studies, including standard forms of meta-analysis. We show that none of these methods satisfies MAP. Thus an underlying measurement problem remains. We argue that a successful approach to evidence amalgamation necessitates a solution to the problem of evidence measurement, and we suggest some lines of reasoning that might guide further work towards this end. (shrink)

Statistical analysis is often used to evaluate the strength of evidence for or against scientific hypotheses. Here we consider evidence measurement from the point of view of representational measurement theory, focusing in particular on the 0-points of measurement scales. We argue that a properly calibrated evidence measure will need to count up from absolute 0, in a sense to be defined, and that this 0-point is likely to be something other than what one might have expected. This suggests the need (...) for a new theory of statistical evidence in the context of which calibrated evidence measurement becomes tractable. (shrink)

Hacking’s Law of Likelihood says—paraphrasing—that data support hypothesis H1 over hypothesis H2 whenever the likelihood ratio for H1 over H2 exceeds 1. But Hacking later noted a seemingly fatal flaw in the LR itself: it cannot be interpreted as the degree of “evidential significance” across applications. I agree with Hacking about the problem, but I do not believe the condition is incurable. I argue here that the LR can be properly calibrated with respect to the underlying evidence, and I sketch (...) the rudiments of a methodology for so doing. (shrink)

Hacking’s Law of Likelihood says – paraphrasing– that data support hypothesis H1 over hypothesis H2 whenever the likelihood ratio for H1 over H2 exceeds 1. But Hacking noted a seemingly fatal flaw in the LR itself: it cannot be interpreted as the degree of “evidential significance” across applications. I agree with Hacking about the problem, but I don’t believe the condition is incurable. I argue here that the LR can be properly calibrated with respect to the underlying evidence, and I (...) sketch the rudiments of a methodology for so doing. (shrink)